Add Azure Storage CLI connection config to specs and yml.erb files #574
Conversation
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
from
45491ee to
6571de4
Compare
- Add four job templates to api, worker, and clock to emit storage-cli JSON: storage_cli_config_droplets.json.erb storage_cli_config_packages.json.erb storage_cli_config_buildpacks.json.erb storage_cli_config_resource_pool.json.erb Each template builds "connection_config" for AzureRM only, applies a default put_timeout_in_seconds: "41" if missing, and passes through any custom flags. - Expose new config paths in cloud_controller_*/*.yml.erb - Register the new files in job specs so they render under config/
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
from
6651acf to
ac99f38
Compare
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
2 times, most recently
from
f4e7f8a to
3b8be47
Compare
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
from
3b8be47 to
0c12f66
Compare
There was a problem hiding this comment.
This is a lot of duplicated code. Can you try to move the common coding to the shared_job_templates folder? I'm not 100% sure if that works, but there should be an option to move common .erb coding to a shared file.
There was a problem hiding this comment.
Could be that it is not so trivial/possible: BOSH renders .erb on the director/agent before the VM’s job tree exists. At this moment, paths like /var/vcap/jobs/cloud_controller_ do not exist. There’s no supported way to “reach over” to shared_job_templates in the repo during render.
…ds is not set, return nil
Co-Authored-By: johha <[email protected]>
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
3 times, most recently
from
82a2ad2 to
b68bb41
Compare
…t, CCNG reads provider from JSON
kathap
force-pushed
the
azure-storage-cli-connection-config
branch
from
b68bb41 to
d7b2b39
Compare
| description: "User's password used to access internal endpoints of Cloud Controller to upload files when staging" | ||
|
|
||
| cc.resource_pool.blobstore_provider: | ||
| description: "The provider of blobstore storage cli to use. Valid values: ['AzureRM']" |
There was a problem hiding this comment.
"AzureRM" originates from the fog-azure implementation. Should we use a different name? Maybe "azure-storage-cli"?
There was a problem hiding this comment.
Or keep AzureRM = Azure Resource Manager
| chown -R vcap:vcap "$LOG_DIR" | ||
| } | ||
|
|
||
| write_blobstore_scope_configs() { |
There was a problem hiding this comment.
I would simply name this write_blobstore_configs()
| write_blobstore_scope_configs() { | ||
| local cfg_json="${CONFIG_DIR}/blobstore_configs.json" | ||
| if [[ ! -f "$cfg_json" ]]; then | ||
| echo "blobstore_configs.json not found at $cfg_json; skipping fan-out" |
There was a problem hiding this comment.
Can this really happen?
| fi | ||
|
|
||
| # Use the packaged Ruby if you prefer; system Ruby is usually fine on CF VMs. | ||
| CFG="$cfg_json" DEST="$BUNDLER_DIR" ruby <<'RUBY' |
There was a problem hiding this comment.
BUNDLER_DIR is nowhere set?
…fig" This reverts commit 0132cad.
|
I realised this branch contains some wrong merge commits. |
2 participants
A short explanation of the proposed change:
Add Azure Storage CLI connection configuration property to the Cloud Controller specs for bosh-azure-storage-cli based blobstore client integration
An explanation of the use cases your change solves
bosh-azure-storage-cli based blobstore client is a potential replacement for deprecated fog libraries like azure fog
Links to any other associated PRs
Extend bosh-azure-storage-cli: new blob commands + upload timeout bosh-azure-storage-cli#22
ADR: Use Bosh Storage CLIs for Blobstore Operations cloud_controller_ng#4443
POC: bosh-azure-storage-cli based blobstore client cloud_controller_ng#4397
I have viewed signed and have submitted the Contributor License Agreement
I have made this pull request to the
developbranchI have run CF Acceptance Tests on bosh lite